Summary

The Jazz History Database Virtual Museum is a virtual reality (VR) museum experience featuring content from the Jazz History Database (JHDB). The primary goal of this project is to create a space to welcome those who might otherwise feel that the world of jazz is arduous to enter or intimidating, help record the history of jazz, and encourage curiosity to explore and experience the world of jazz interactively. The Jazz History Database Virtual Museum was developed using Unreal Engine 5 for the HTC VIVE VR headset and Oculus Quest; it is available for Windows. We completed this Major Qualifying Project (MQP) as part of the Interactive Media and Game Development program.

User research and Market research

Understand the Sponsor - JHDB

The Jazz History Database (JHDB) is a non-profit organization at Worcester Polytechnic Institute with the mission to archive recordings, interviews, photographs, and other jazz artifacts from around the world. The goal is to celebrate artists’ accomplishments in music, poetry, visual art, or any other field. Visit JHDB Website here!

Experience Goals

We aimed to establish a space somewhere between a jazz club and a museum. We wanted the space tointegrate jazz elements and invite users to discover and familiarize themselves with various jazz-relatedmedia. At the same time, we also wanted it to provide a new experience for jazz enthusiasts. Sincethis is a museum experience rather than a game, we wanted it to be as easy to control and navigateas possible, even for a beginner with no VR experience. Furthermore, we wanted to create a uniqueactive listening experience and emphasize the instrumental aspects of jazz.

Target Audience

We wanted to create an experience for people curious about jazz and offer a virtual space for those who are already familiar with it but want to experience it in a new way. The target audience in the U.S. primarily appeals to adults aged 35+ (De Novo Agency, 2023). As a result, many people unfortunately associate the image of jazz with the older generation and traditional values. Moreover, jazz is often described as complex and sophisticated and often appears in high-end places with agerestrictions (MasterClass, 2021; Schuller, 2023). These facts can make jazz seem challenging to theyounger generation. The primary audience for jazz music may be the older generation, but this does not imply that younger groups are indifferent to it. This museum aims to cater to both mature and younger audiences and to provide them both with an opportunity to appreciate and enjoy jazz.

In our conversation with Ben Young, our sponsor, we discussed the target audiences of jazz andthe virtual museum. He identified that he wanted the project to appeal to people who want a jazz experience first. He also wants to attract a general population into the VR experience that can be influenced to enter the jazz world.

Inspiration

Our sponsor, Ben Young, believes that the most important task in jazz is listening. With that in mind, we wanted to create a space where the act of listening is the main focus. As the great Miles Davis said: "It’s just sound. One sound leads to another."

Paradiddle VR is one of the best combinations of music and VR. It is a virtual game where players can set up drum sets and play. This game allows users to interact at their own paces and try out a separate set of instruments to create sounds. We want to implement similar aspects to this experience

We also took inspiration from real-life museums. Several museums celebrate and preserve the history and culture of jazz music in the United States. Some of them are local museums in Worcester were also referenced in the process of designing this project, including the Worcester Historical Museum and Worcester Art Museum. Worcester's El Morroco was a hotspot for Jazz enthusists back in the days, this was also the inspiration for us to create two different spaces.

Artistic Implementation

The environment and prop production pipeline began with looking into last year’s project and gathering reference images for the type of space the team wanted to create. Last year’s project implemented as peak-easy space where the users can listen to music by changing the records on the shelf.Multiple ideas were collected and discussed among team members on which ideas to move on. During the first stage, none of the ideas were finalized and were all put into the draft box. Nevertheless, we all agreed that the space must include a stage where users can interact with instruments and seethe band layout. Having a stage with musical instruments serves the purpose of showcasing live music performances by jazz artists and bands as well as the instruments themselves.

We experimented with a few layouts and selected the one in the image above for the museum’s first iteration.

The greyboxing process enabled us to create and test a playable experience. It also helped us to comprehend the scale of objects in VR. Through this design, we were able to prioritize the assets that we required. Moreover, it saved us time as we did not have an artist and could focus on the technical elements first and address the artistic implementation later.

With the layout we achieved from the greyboxing process, we have two options: an orchestra or a small, closed room space as a viable layout for jazz. We recreated a small, closed environment - a cabaret with a stage for a jazz performance. As stated in Artistic Inspiration, we were curious about how people came across jazz daily. By using this idea, we can make jazz seem less intimidating and provide a situation that would mimic real-life exposure to jazz. This idea also allowed us to recreate some historical context of jazz. With this idea in mind, we created the project's second implementation, as shown below!

Our goal for the final iteration was to maintain the stage as the focal point and expand the environment to create more room for interaction and display the JHDB content. At the same time, we wanted to achieve a jazz club's lively atmosphere and the museum's educational value; this led to our decision to have two separate environments for users to explore. We believed that having two different designs of the museum could first differentiate the purpose of the space while also making sure that what we intended to stand out and highlight stands out in the environment. So, in our first environment, users can interact with interviews and view the facts of an artifact similar to a museum. In the second environment, we focus more on the sound of jazz and the liveliness, the lighting, and the atmosphere of a jazz club. Another reason for having two different designs is to allow our users to accommodate different populations' needs and tastes. This does not change the core functionality or the project's goals but gives users space to explore more and be creative. Lastly, we also believe that having two separate spaces could be cultural adaptations that represent what the jazz history database is willing to modify to suit the current market norm and respect jazz's cultural value.

Technical Implementation

Designing for VR Interaction

For a VR experience, we need to take into consideration user inputs. In VR, there are multiple ways to accomplish movements: movement tracking in the room-scale capable headset, using a trackpad or joystick to enable locomotion to move, or using a button for teleportation in the environment. To ensure usability and accessibility across platforms, we implement all three movement methods in the Virtual Museum. We targeted the museum as an experienced user can take home, so usability is critical. Usability affects the users' comfort, safety, and satisfaction, so we should design a VR application to avoid user fatigue, dizziness, neck pain, and other potential risks.

Spatial Movement

There are two ways to move around the environment in the Virtual Museum: teleportation and continuous locomotion. To teleport, the user holds the trigger on either controller. An arc will appear from that hand, and when the player releases the trigger, they will teleport to the location that the arc touches on the floor. Alternatively, the player can use the thumbstick or trackpad on either controller to move throughout the space continuously as if they are walking. We decided to implement both types of locomotion. That way, users could decide how to move around the space and opt entirely out of continuous locomotion if it caused motion sickness.

Grab and Release

The Virtual Museum's grab-and-release functionality is designed to make it easy to specify which objects can be grabbed and which cannot. To make an object grabbable, it simply needs to implement GrabbableInterface. From there, the object can define unique behavior when it is grabbed and released. This subsection describes the current implementation of GrippbleInterface. Moreover, how the VrCharacter handles grabbing. This functionality allows users to actually interact with recreated artifacts from the JHDB without damaging the real displays.

VR Camera

The camera for VR mode is attached to the VR character. The Field of View is set to 130 degrees to replicate the field of view of humans in real life. The camera’s height is configured by the height of users so that users can move smoothly inside the environment.

Inspector Mode

Even though our project is designed for a VR-based environment, we wanted to make the application accessible for other platforms, not just VR. An inspector is implemented to make the application available to use on Windows without a proper VR device. Users can move around the environment using a keyboard and mouse. Unlike normal VR spectators in other VR applications, this spectator is a stand-alone camera allowing users to inspect the environment using first-person view and move around.

There are three main components to the VR Spectator Camera: Screen Texture, Mode, and Blueprint.

The main component to notice is the SceneCaptureComponent2D inside the VRSpectator Blueprint. This SceneCaptureComponent2D is linked to the Target Renderer of the Screen Texture. This means the SceneCaptureComponent2D takes a screenshot of each frame and renders it on the users’ viewport.

The movement of this camera is mapped using typical first-person controls. Use the W, A, S, and D keys to move and use the mouse to look around. To activate Spectator view, users must use the “Tab” key on the keyboard to switch from VRGameMode to VRSpectator view.

Audio Implementation

This section describes the sound design for the JHDB Virtual Museum. As this project aims to connect jazz to more audiences, the job of an audio engineer is to incorporate jazz into the environment and enhance the current experiences. In this experience, sounds create a realistic and immersive virtual museum experience.
The museum required audio that not only fit the aesthetics of the environment but also was engaging and set the atmosphere for the surroundings. This audio implementation worked closely with the development of technical and artistic elements of the museum to make the experience as immersive as possible.

Sound Effects

Sound Effects in the JHDB Virtual Museum broke down into three main effects: footsteps for locomotion, grabbing/drop sounds, and musical instrument sound effects. Most sounds were initially sourced from freesound.org, a website where anyone can download free sound assets. Some sounds were extracted from the tracks sent to us from the JHDB. The sound effect process consisted of downloading or extracting the sounds. Then, we cut and edited the sounds, equalizing them and removing noise. Lastly, we converted them to WAV because this is the only audio file type Unreal Engine 5 supports without plug-ins.

Ambiance & Music

The JHDB representative, Ben Young, gave us the original background music. However, if we only have music in the background, the experience might appear flat and one-dimensional to users. To make it immersive, we use spatial sound provided by Unreal to make the experience as realistic as possible. A mix of a jazz soundtrack and ambiance sounds like a dine-in live entertainment establishment or museum ambiance was used to mimic the atmosphere at a contemporary museum or a restaurant.

User Testing & Analysis

Throughout the development process, the team conducted two surveys, one as secondary research on the audiences and one user usability section during the IMGD-sponsored AlphaFest event. The first was to understand our user's perspective; the second was to measure the overall accessibility and experience for improvement in the future. The playtest session concluded with an anonymous survey of users.

The details of this user experience research are recorded and explained in this report.

Conclusion

This project initially set out to create a virtual reality museum experience that could help promote and preserve the content of the Jazz History Database. The primary goal of this project is to create a space to welcome those who might otherwise feel that the world of jazz is arduous to enter or intimidating in a way, record the history of jazz, and encourage curiosity to explore and experience the world of jazz more interactively.
Overall, the entire team felt that this MQP was a success. Even though we did not have a dedicated artist, we achieved two different environment designs that conveyed our ideas well. From the technical standpoint, we provided users with an adequate level of interaction to explore the virtual space with fun and educational intention. Based on the results and feedback the team received in person and through written comments, the team felt the target experience goal was met and that this MQP was a project to be proud of.